Machine learning model for analyzing pathology data from metastatic sites

ABSTRACT

Described herein are systems and methods of determining primary sites from biomedical images. A computing system may identify a first biomedical image of a first sample from one of a primary site or a secondary site associated with a condition in a first subject. The computing system may apply the first biomedical image to a site prediction model comprising a plurality of weights to determine the primary site for the condition. The computing system may store an association between the first biomedical image and the primary site determined using the site prediction model.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application claims priority under 35 U.S.C. § 120 as acontinuation of U.S. patent application Ser. No. 17/335,925, titled“Machine Learning Model for Analyzing Pathology Data from MetastaticSites,” filed Jun. 1, 2021, which claims priority under 35 U.S.C. §119(e) to U.S. Provisional Application No. 63/033,730, titled “MachineLearning Model for Analyzing Pathology Data from Metastatic Sites,”filed Jun. 2, 2020, each of which is incorporated herein by reference intheir entirety.

BACKGROUND

A computing device may use various computer vision algorithms torecognize and detect various features in images. The computing devicemay also determine characteristics regarding the features within theimages.

SUMMARY

Aspects of the present disclosure are directed to systems and methods ofdetermining primary sites from biomedical images. A computing system mayidentify a first biomedical image of a first sample from one of aprimary site or a secondary site associated with a condition in a firstsubject. The computing system may apply the first biomedical image to asite prediction model comprising a plurality of weights to determine theprimary site for the condition. The site prediction model may be trainedusing a training dataset having a plurality of examples. Each examplemay include: a second biomedical image of a second sample from one ofthe primary site or the secondary site of the condition in a secondsubject from which the second sample is obtained, a first labelidentifying one of the primary site or the secondary site for the secondsample from which the second biomedical image is obtained, and a secondlabel identifying the primary site for the condition in the secondsubject. The computing system may store an association between the firstbiomedical image and the primary site determined using the siteprediction model.

In some embodiments, the computing system may provide the associationbetween the first biomedical image and the primary site. In someembodiments, the computing system may apply the first biomedical imageto the site prediction model to determine a plurality of candidateprimary sites for the condition in the first subject.

In some embodiments, the computing system may apply the first biomedicalimage to the site prediction model to determine a confidence score forthe primary site for the condition. In some embodiments, the computingsystem may apply the first biomedical image to the site prediction modelto determine a ranking for the plurality of candidate primary sites.

In some embodiments, the computing system may obtain the firstbiomedical image of the first sample via a histological image preparer.In some embodiments, the plurality of weights in the site predictionmodel may be arranged into (i) a plurality of convolution blocks togenerate a plurality of feature maps from the biomedical image and (ii)an activation layer to determine the primary site for the conditionbased on the plurality of feature maps.

Aspects of the present disclosure are directed to systems and methods oftraining models to determine primary sites from biomedical images. Acomputing system may identify a training dataset having a plurality ofexamples. Each example of the plurality of examples may include: abiomedical image of a sample from one of the primary site or thesecondary site of the condition in a second subject from which thesample is obtained, a first label identifying one of the primary site orthe secondary site for the second sample from which the biomedical imageis obtained, a second label identifying the primary site for thecondition in the second subject. The computing system may apply thebiomedical image in each of the plurality of examples of the trainingdataset to a site prediction model comprising a plurality of weights todetermine a site for the condition of the sample. The computing systemmay compare, for each example of the plurality of examples in thetraining dataset, the primary site identified in the label of theexample and the site determined by the site prediction model. Thecomputing system may update at least one of the plurality of weights inthe site prediction model based on the comparison between the first siteidentified in the label of each example and the second site determinedby the site prediction model. The computing system may store, in one ormore data structures, the plurality of weights in the site predictionmodel.

In some embodiments, the computing system may apply an acquiredbiomedical image of a second sample to the site prediction model todetermine a second site for the second sample. In some embodiments, thecomputing system may reapply the biomedical image of at least oneexample of the plurality of examples to the site prediction model,responsive to determining that a loss metric for the at least oneexample exceeds a threshold.

In some embodiments, the computing system may update at least one of theplurality of weights in the site prediction model using a classificationloss determined based on the comparison. In some embodiments, thecomputing system may apply the biomedical image in each of the pluralityof examples of the training dataset to determine a plurality ofcandidate primary sites for the condition in the first subject.

In some embodiments, the computing system may apply the biomedical imagein each of the plurality of examples of the training dataset todetermine a confidence score for the primary site for the condition. Insome embodiments, the plurality of weights in the site prediction modelmay be arranged into (i) a plurality of convolution blocks to generate aplurality of feature maps from the biomedical image and (ii) anactivation layer to determine the primary site for the condition basedon the plurality of feature maps.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, aspects, features, and advantages ofthe disclosure will become more apparent and better understood byreferring to the following description taken in conjunction with theaccompanying drawings, in which:

FIGS. 1A-1 to 1A-7 depict a block diagram of an overview of a processfor identification of a primary site of origin in metastatic sites usingwhole slide images in accordance with an illustrative embodiment;

FIGS. 1B-1 and 1B-2 depict a block diagram of an overview of a processfor training a model for identification of a primary site of origin inmetastatic sites using whole slide images in accordance with anillustrative embodiment;

FIG. 1C depicts a graph showing relations between primary sites andmetastatic sites in training data used to train the model foridentification of primary site of origin in metastatic sites inaccordance with an illustrative embodiment;

FIG. 2 depicts a block diagram of a system for determining primary sitesfrom biomedical images in accordance with an illustrative embodiment;

FIG. 3 depicts a block diagram of a training process for a siteprediction model in a system for determining primary sites frombiomedical images in accordance with an illustrative embodiment;

FIG. 4A depicts a block diagram of an architecture of a site predictionmodel in a system for determining primary sites from biomedical imagesin accordance with an illustrative embodiment;

FIG. 4B depicts a block diagram of an architecture of an encoder blockin a site prediction model of a system for determining primary sitesfrom biomedical images in accordance with an illustrative embodiment;

FIG. 4C depicts a block diagram of an architecture of a transform stackin a site prediction model of a system for determining primary sitesfrom biomedical images in accordance with an illustrative embodiment;

FIG. 5 depicts a block diagram of an inference process for a siteprediction model in a system for determining primary sites frombiomedical images in accordance with an illustrative embodiment;

FIG. 6A depicts a flow diagram of a method of training models todetermine primary sites, in accordance with an illustrative embodiment;

FIG. 6B depicts a flow diagram of a method of applying models todetermine primary sites, in accordance with an illustrative embodiment;and

FIG. 7 depicts a block diagram of a server system and a client computersystem in accordance with an illustrative embodiment.

DETAILED DESCRIPTION

Following below are more detailed descriptions of various conceptsrelated to, and embodiments of, systems and methods for determiningprimary sites from biomedical images. It should be appreciated thatvarious concepts introduced above and discussed in greater detail belowmay be implemented in any of numerous ways, as the disclosed conceptsare not limited to any particular manner of implementation. Examples ofspecific implementations and applications are provided primarily forillustrative purposes.

Section A describes systems and methods for determining primary sitesfrom biomedical images; and

Section B describes a network environment and computing environmentwhich may be useful for practicing various computing related embodimentsdescribed herein.

A. Systems and Methods of Determining Primary Sites of Biomedical Images

Cancerous cells may originate from a primary site and spread to one ormore secondary sites (also referred herein as metastatic sites)throughout a body of a subject. Depending on the tumor site, thecancerous cells at the secondary site may appear characteristicallysimilar to those in the primary site of originate. An estimate of theprimary site may be a critical factor in devising treatments toalleviate and suppress the spread of cancer in the body. A pathologistmay manually examine a whole slide image of tissues from the subject todetermine the primary site of origin for the cancer. This process,however, may be slow and cumbersome especially when large numbers ofimages from multiple subjects are to be examined. Furthermore, themanual nature of the examination may lead to inaccurate diagnoses,resulting in incorrect treatment recommendations. One approach toaccount for some of these problems may be to use computer visiontechniques to recognize cancerous cells within a given tissue depictedin the whole slide image. But this approach may fail to address atproviding an estimate of the primary site for the cancerous cells. Toaddress these and other challenges, a model may be trained to learnmorphological patterns to predict primary sites of cancerous cells basedon whole slide images. The model may identify morphological patternsthat are highly correlated with metastatic risk.

Referring now to FIG. 1A, depicted is a block diagram of an overview ofa process 100 for identification of a primary site of origin inmetastatic sites using whole slide images. As depicted, ground truth maybe automatically extracted from data records of pathology records. Thepath reports may include tumor types, such as gradation (e.g., highgrade osteosarcoma, low-grade serious carcinoma, or high-gradepleomorphic sarcoma), differentiation (e.g., poorly differentiatedthyroid carcinoma, undifferentiated pleomorphic sarcoma,well-differentiated mucinous adenocarcinoma, and de-differentiatedliposarcoma), tumor (e.g., neuroblastic tumor, Muellerian tumor, andsmall-round cell tumor), sub-tumor types (e.g., Ewing sarcoma,angiosarcoma, and retinoblastoma), characteristics (e.g., invasivecarcinoma, malignant tumor, and metastasizing leiomyoma), and celltumors or abnormalities (e.g., squamous cell carcinoma, germ cell tumor,clear cell carcinoma, and non-small cell carcinoma), among others. Adataset of whole slide images (WSI) indexed by tumor types may becompiled from several subjects. The tumor regions may be extracted undermarker extraction, pathologist annotations, and class activation maps.Using the relevant regions, a classifier may be trained to locate theprimary site for the tumor depicted in the image. Additional metastaticrisk factors may be analyzed with the prediction of the primary sitesfrom the tissue sample.

Referring now to FIG. 1B, depicted is a block diagram of an overview aprocess 105 for training a model for identification of a primary site oforigin in metastatic sites using whole slide images. As depicted, amodel may be trained using training data in a self-supervised manner.The model may be trained to recognize the tumor types. Markerannotations as well as pathologist annotations may be used to detect thelocation of the tumorous cell within the whole slide image. Referringnow to FIG. 1C, depicted is a graph 110 showing relations betweenprimary sites and metastatic sites in training data used to train themodel for identification of primary site of origin in metastatic sites.As depicted, there may be a correlation between primary sites of cancerand metastatic sites. The model trained using these data may developweights and connections to infer the correlations for new input images.

Referring now to FIG. 2 , depicted is a block diagram of a system 200for determining primary sites from biomedical image. In overview, thesystem 200 may include at least one image processing system 205, atleast one imaging device 210, and at least one display 215communicatively coupled with one another via at least one network 220.The image processing system 205 may include at least one model trainer225, at least one model applier 230, at least one site prediction model235, and at least one database 240, among others. The database 240 maystore, maintain, or otherwise include at least one training dataset 245.Each of the components in the system 200 as detailed herein may beimplemented using hardware (e.g., one or more processors coupled withmemory) or a combination of hardware and software as detailed herein inSection B.

In further detail, the image processing system 205 itself and thecomponents therein, such as the model trainer 225, the model applier230, and the site prediction model 235, may have a training mode and aruntime mode (sometimes herein referred to as an evaluation orinterference mode). Under the training mode, the image processing system205 may invoke the model trainer 225 and the model applier 230 to trainthe site prediction model 235 using the training dataset 245. Under theruntime mode, the image processing system 205 may invoke the modelapplier 230 to apply the site prediction model 235 to new biomedicalimages to predict a primary site of a condition in a subject whosetissue is depicted in the biomedical image.

Referring now to FIG. 3 , depicted is a block diagram of a trainingprocess 300 for the site prediction model 235 in the system 200 fordetermining primary sites from biomedical images. The process 300 maycorrespond to or include the operations performed by the imageprocessing system 205 under the training mode. Under the process 300,the model trainer 230 executing on the image processing system 205 mayinitialize, train, or establish the site prediction model 235. In someembodiments, the model trainer 230 may assign random values to set ofweights in the site prediction model 235 as part of the initialization.To train the site prediction model 235, the model trainer 225 may accessthe database 240 to retrieve, fetch, or identify the training dataset245. The training dataset 245 may be stored and maintained on thedatabase 240 using at least one data structure (e.g., an array, amatrix, a heap, a list, a tree, or a data object). With theidentification, the model trainer 225 may train the site predictionmodel 235 using the training dataset 245. The training of the siteprediction model 235 may be in accordance with supervised (e.g., activeor weakly supervised) learning techniques.

The training dataset 325 may include one or more examples. Each exampleof the training dataset 245 may include at least one image 305, at leastone primary site label 310, and at least one image site label 315, amongothers from a subject 320. The examples in the training dataset 325 maybe obtained from multiple subjects 325. In each training dataset 245,the image 305 may be acquired, derived, or otherwise may be of at leastone sample 325 from the subject 320. The sample 325 may be a tissuesection taken or obtained from the subject 320 (e.g., a human, animal,or flora). The tissue section may include, for example, a muscle tissue,a connective tissue, an epithelial tissue, nervous tissue, or an organtissue, in the case of a human or animal subject. The sample 325 may beobtained from at least one primary site 330 or at least one secondarysite 335 (sometimes referred herein as a metastatic site) of the subject320. The sample 325 itself may have or include one or more objects withthe conditions. For example, the tissue section of the sample 325 maycontain tumorous cells or lesions thereon. In this example, thecancerous cells or the lesion may correspond to the object and thecondition may correspond to having a tumor or lesion. The primary site330 may correspond to a location in the subject 320 from which thecondition originated. The secondary site 335 may correspond to alocation in the subject 320 to which the condition spread. For example,for a subject 320 with lung cancer that spread to the brain, the primarysite 330 may be a location within the lung and the secondary site 335may be a location within the brain.

The image 305 itself may be acquired in accordance with microscopytechniques or a histopathological image preparer, such as using anoptical microscope, a confocal microscope, a fluorescence microscope, aphosphorescence microscope, an electron microscope, among others. Theimage 305 may be, for example, a histological section with a hematoxylinand eosin (H&E) stain, immunostaining, hemosiderin stain, a Sudan stain,a Schiff stain, a Congo red stain, a Gram stain, a Ziehl-Neelsen stain,a Auramine-rhodamine stain, a trichrome stain, a Silver stain, andWright's Stain, among others. The image 305 may include one or moreregions of interest (ROIs). Each ROI may correspond to areas, sections,or boundaries within the sample image 305 that contain, encompass, orinclude conditions (e.g., features or objects within the image). Forexample, the sample image 305 may be a whole slide image (WSI) fordigital pathology of a tissue section in the sample 325, and the ROIsmay correspond to areas with lesions and tumors in the sample tissue. Insome embodiments, the ROIs of the sample image 305 may correspond todifferent conditions. Each condition may define or specify aclassification for the ROI. For example, when the image 305 is a WSI ofthe sample tissue, the conditions may correspond to varioushistopathological characteristics, such as carcinoma tissue, benignepithelial tissue, stroma tissue, necrotic tissue, and adipose tissue,among others. In some embodiments, the training dataset 245 may includeat least one annotation identifying the ROIs in the associated image305.

In addition, the primary site label 310 may identify the primary site330 in the subject 320 from which the condition in the sample 325originated. The image site label 315 may identify a site in the subject320 from which the sample 325 is obtained. Both the primary site label310 and the image site label 315 may contain a value (e.g., alphanumericor numeric) corresponding to one of potential sites in the subject 320.The image site label 315 may be the primary site 330 or the secondarysite 335. The primary site label 310 and the image site label 315 maydiffer or may be the same. When the image 305 is of a sample 325obtained from the primary site 330, the primary site label 310 and theimage site label 315 may be the same. In this case, both the primarysite label 310 and the image site label 315 may identify the primarysite 330. When the image 305 is of a sample 325 obtained from thesecondary site 335, the primary site label 310 and the image site label315 may differ. In this case, the primary site label 310 may identifythe primary site 330 and the image site label 315 may identify thesecondary site 335 in the subject 320. The primary site label 310 andthe image site label 315 may be inputted or generated by a pathologistor clinician examining the subject 320 or the sample 325. In someembodiments, the image site label 315 may be omitted from the trainingdataset 245.

In training, the model applier 230 executing on the image processingsystem 205 may apply the image 305 from the training dataset 245 to thesite prediction model 230. The site prediction model 235 may include orhave a set of weights (sometimes herein referred to as parameters,kernels, or filters) to process at least one input and produce at leastone output. The set of weights in the site prediction model 235 may bearranged, for example, in accordance with a convolutional neural network(CNN) architecture, such as an array of ResNet-50 CNNs. In applying, themodel applier 230 may provide or feed the image 305 from each example ofthe training dataset 245 to the input of the site prediction model 235.In some embodiments, the model applier 230 may feed the entirety of theimage 305 to the site prediction model 235. In some embodiments, themodel applier 230 may select or identify one or more tiles from theimage 305 to input to the site prediction model 235.

Upon feeding, the model applier 230 may process the input image 305 inaccordance with the set of weights arranged in the site prediction model235 to generate at least one output. The output may include one or morepredicted primary sites 340. Each predicted primary site 340 mayidentify the primary site 330 for the condition depicted in the sample325 of the input image 305. The predicted primary site 340 may include avalue (e.g., alphanumeric or numeric) corresponding to one of the sites(e.g., organ) in the subject 320. In some embodiments, the output mayinclude a confidence score for each predicted primary site 340 for thecondition. The confidence score may define or indicate a degree oflikelihood that the predicted primary site 340 for the condition is theactual primary site 330 for the condition. Details of the architectureand functioning of the site prediction model 235 are described hereinbelow in conjunction with FIGS. 4A-C.

Referring now to FIG. 4A, depicted is a block diagram of an architecture400 of the site prediction model 235 in the system 205 for determiningprimary sites from biomedical images. Under the architecture 400, thesite prediction model 235 may include one or more encoders 405A-N(hereinafter generally encoders 405), at least one aggregator 410, andat least one activator 415 (sometimes herein generally referred to as anactivation layer), among others. The set of weights of the siteprediction model 235 may be configured, arrayed, or otherwise arrangedacross the one or more encoders 405, the aggregator 410, and theactivator 415, among others. The site prediction model 235 may have oneor more inputs and at least one output. The input may include the image305 or a set of tiles 420A-N (hereinafter generally referred to as tiles420) from the image 305. The tile 420 may correspond to a portion of theimage 305. The output may include the predicted primary site 340 (e.g.,as depicted). In some embodiments, the output may include the confidencescore for the predicted primary site 340. The inputs and outputs of theencoders 405, the aggregator 410, and the activator 415 may be connectedwith one another, for example, in the manner depicted.

Each encoder 405 may receive, retrieve, or otherwise identify at least aportion of the image 305 as the input. The input may be the entirety ofthe image 305 or a corresponding tile 420 from the image 305. Inaccordance with the weights in the encoder 405, the encoder 405 mayprocess the input. The set of weights in the encoder 405 may bearranged, for example, according to a convolutional neural network(CNN). In some embodiments, the set of weights may be shared amongst theencoder 405. For example, the values and interconnections of the weightswithin the encoder 405 may be the same throughout the encoders 405 inthe site prediction model 235. In some embodiments, the set of weightsmay be not shared among the encoders 405. For instance, the values orthe interconnections of the weights in one encoder 405 may differ or maybe independent of the values or the interconnections of the weights inother encoders 405. The encoder 405 may be implemented using thearchitectures detailed herein in conjunction with FIGS. 4B and 4C. Fromprocessing the input, the encoder 405 may produce or generate at leastone feature map 425A-N (hereinafter generally referred to as featuremaps 425). The feature map 425 may be a lower dimensional representationof the input image 305 or tile 420. For example, the feature map 425 maybe a representation of latent features in the input image 305 orrespective tile 420. The output of encoder 405 may be provided or fed asthe input of the aggregator 410.

The aggregator 410 may in turn receive, retrieve, or otherwise identifythe feature maps 425 generated by the corresponding encoders 405. Uponreceipt, the aggregator 410 may concatenate or combined the feature maps425 for input into the set of weights defined in the aggregator 410. Theaggregator 410 may process the input in accordance with the set ofweights. In some embodiments, the set of weights in the aggregator 410may be arranged according to a fully convolutional network (FCN). Theaggregator 410 may be implemented using the architectures detailedherein in conjunction with FIGS. 4B and 4C. By processing, theaggregator 410 may determine, produce, or otherwise generate at leastone aggregate feature map 430. The aggregate feature map 430 may be alower dimensional representation of the combined set of received featuremaps 425. For instance, the aggregate feature map 430 may be arepresentation of latent features in the combined set of feature maps425 from the encoders 405. The output of the aggregator 410 may beprovided or fed as the input of the activator 415.

The activator 415 may receive, retrieve, or otherwise identify theaggregator feature map 430 generated by the aggregator 401. Theactivator 415 may process the input aggregate feature map 430 inaccordance with the set of weights. The set of weights in the activator415 may be arranged according to an activation layer, such as a softmaxfunction, a maxout function, a rectified linear unit (ReLU), a linearactivation function, a heavyside function, a radial function, or alogistic function, among others. The activator 415 may be implementedusing the architectures detailed herein in conjunction with FIGS. 4B and4C. From processing, the activator 415 may produce or generate at leastone output. The output may include at least one predicted primary site340. The predicted primary site 340 may correspond to a defined site inthe subject 320, such as the lung, breast, brain, liver, stomach,thyroid, skin, or any other organ, among others. In some embodiments,the output may include the confidence score for the predicted primarysite 340. The confidence score may define or indicate a degree oflikelihood that the predicted primary site 340 for the condition is theactual primary site 330 for the condition.

Referring now to FIG. 4B, depicted is a block diagram of an architecture440 of an encoder block 445 in the site prediction model 235 of thesystem 200 for determining primary sites from biomedical images. Theencoder block 445 may be used to implement the individual encoders 405as well as the aggregator 410 in the site prediction model 245. Forexample, each encoder 405 and the aggregator 410 may be an instance ofthe encoder block 445. Under the architecture 440, the encoder block 445may include one or more convolution stacks 450A-N (hereinafter generallyreferred to as convolution stacks 450). The encoder block 415 may alsoinclude at least one input 455 and at least one output such as a featuremap 460. The input 455 and the output feature map 460 may be related viathe set of weights defined in the convolution stacks 450. When used toimplement the encoder 405, the input 455 of the encoder block 445 maycorrespond to or include the image 305 or the corresponding tile 420 andthe output feature map 460 may correspond to the feature map 425. Whenused to implement the aggregator 410, the input 455 of the encoder block445 may correspond to or include the combined set of feature maps 425and the output 460 may correspond to the aggregate feature map 430. Eachconvolution stack 450 may define or include the weights the encoderblock 445. The set of convolution stacks 450 can be arranged in series(e.g., as depicted) or parallel configuration, or in any combination. Ina series configuration, the input of one convolution stacks 450 mayinclude the output of the previous convolution stacks 450 (e.g., asdepicted). In parallel configuration, the input of one convolutionstacks 450 may include the input of the entire encoder lock 445. Detailsregarding the architecture of the convolution stack 450 are providedherein below in conjunction with FIG. 4C.

Referring now to FIG. 4C, depicted is a block diagram of an architecture470 of a transform stack 475 in the site prediction model 235 of thesystem 200 for determining primary sites from biomedical images. Thetransform stack 475 may be used to implement the convolution stacks 450of the encoder blocks 445 used as instances of encoders 405 oraggregators 410 in the site prediction model 235. The transform stack475 may also be used to implement the activator 415 in the predictionmodel 235. The transform stack 475 may include one or more transformlayers 480A-N (hereinafter generally referred to as transform layers480). The transform stack 475 also include at least one input 485 and atleast one output feature map 490. The input 485 and the output 490 maybe related via the set of weights defined in the transform layers 480 ofthe transform stack 475. When used to implement the activator 415, theinput 485 may correspond to the aggregate feature map 430 and the output490 may correspond to the predicted primary site 340 and the confidencescore, among others. The set of transform layers 480 can be arranged inseries, with an output of one transform layer 480 fed as an input to asucceeding transform layer 480. Each transform layer 480 may have anon-linear input-to-output characteristic. The transform layer 480 maycomprise a convolutional layer, a normalization layer, and an activationlayer (e.g., a rectified linear unit (ReLU)), among others. In someembodiments, the set of transform layers 480 may be a convolutionalneural network (CNN). For example, the convolutional layer, thenormalization layer, and the activation layer (e.g., a softmax function)may be arranged in accordance with CNN.

In the context of FIG. 3 , the model trainer 225 may retrieve, obtain,or otherwise identify the output produced by the site prediction model235 from the application of the image 305. The output may include, forexample, the at least one predicted primary site 340 and the confidencescore for the predicted primary site 340, among others. In conjunction,the model trainer 225 may identify the input image 305, the primary sitelabel 310, or the image site label 315 of the example in the trainingdataset 245 used to generate the predicted primary site 340. In someembodiments, the model trainer 225 may identify the primary site label310 identified in the example of the training dataset 245 used togenerate the predicted primary site 340. In some embodiments, the modeltrainer 225 may identify the image site label 315 identified in the sameexample.

With the identification, the model trainer 225 may compare the predictedprimary site 340 generated by the site prediction model 235 with theprimary site label 310 as identified in the example from the trainingdataset 245. When there are multiple predicted primary sites 340outputted by the site prediction model 235, the model trainer 225 mayselect or identify the predicted primary site 340 with the highestconfidence score for training purposes. In some embodiments, the modeltrainer 225 may compare the value included in the predicted primary site340 with the value indicated by the primary site label 310.

From comparing, the model trainer 225 may determine whether thepredicted primary site 340 generated by the site prediction model 235 iscorrect. When the predicted primary site 340 matches the primary sitelabel 310, the model trainer 225 may determine that the predictedprimary site 340 is correct. In some embodiments, the model trainer 225may identify the example in the training dataset 245 used to produce thepredicted primary site 340 as correctly determined. The model trainer225 may also exclude the example from the training dataset 245 from aretraining dataset. Conversely, when the predicted primary site 340 doesnot match the primary site label 310, the model trainer 225 maydetermine that the predicted primary site 340 is incorrect. In someembodiments, the model trainer 225 may identify the example in thetraining dataset 245 used to produce the predicted primary site 340 asincorrect determined (sometimes herein referred to as a hard example).The model trainer 225 may include the example from the training dataset245 in the retraining dataset.

In some embodiments, the model trainer 225 may factor in the image sitelabel 315 of the example in the training dataset 325 in includingexamples in the retraining dataset. The model trainer 225 may identify anumber of examples in the retraining dataset with the image site label315 for a particular site in the subject 320. The model trainer 225 maycompare the number of examples to a threshold. The threshold may definea value for the number of examples at which to include all examples withthe same image site label 315 into the retraining dataset. When thenumber of examples is greater than or equal to the threshold, the modeltrainer 225 may include all the examples from the training dataset 245with the same image site label 315 into the retraining dataset.Otherwise, when the number of examples is less than the threshold, themodel trainer 225 may maintain the current number of examples in theretraining dataset.

Based on the comparisons, the model trainer 225 may calculate, generate,or otherwise determine at least one loss metric (sometimes hereinreferred to as an error metric) for updating the weights of the siteprediction model 235. The loss metric may be determined using manyoutputs from the site prediction model 235 generated using many examplesfrom the training dataset 245. The loss metric may indicate a degree ofdeviation of the output (e.g., the predicted primary site 340) from thesite prediction model 235 from the expected result (e.g., the primarysite label 310) as indicated in the training dataset 245. For example,the loss metric may measure a classification loss from incorrectclassifications of the images 305 into one of the sites besides thecorrect site as identified in the primary site label 310. The lossmetric may be calculated in accordance with any number of lossfunctions, such as a Huber loss, norm loss (e.g., L1 or L2), meansquared error (MSE), a quadratic loss, and a cross-entropy loss, amongothers. In general, the higher the loss metric, the more the output mayhave deviated from the expected result of the input. Conversely, thelower the loss metric, the lower the output may have deviated from theexpected result. In some embodiments, an example may be included in theretraining dataset when the loss metric for the predicted primary site340 generated using the example is greater than a threshold. Thethreshold may define or delineate a value for the loss metric at whichto include the corresponding example into the retraining dataset. Insome embodiments, the model trainer 225 may combine the results of thecomparisons with respect to the output and the training dataset 245 tocalculate the loss metric.

Using the loss metric, the model trainer 225 may modify, set, orotherwise update one or more weights in the site prediction model 235.The updating of the weights may be across the encoders 405, theaggregator 410, and the activator 415 within the site prediction model245. The updating of weights may be in accordance with an optimizationfunction (or an objective function) for the site prediction model 235.The optimization function may define one or more rates or parameters atwhich the weights of the site prediction network 235 are to be updated.The updating of the kernels in the site prediction model 235 may berepeated until a convergence condition.

In some embodiments, the model trainer 225 may use the retrainingdataset to continue training the site prediction model 235. Theretraining dataset may include one or examples for which the siteprediction model 235 previously produced an incorrected predictedprimary site 340. The training of the prediction model 235 using theretraining dataset may be similar to training using the original dataset245 as described above. For example, the model applier 230 may reapplythe image 305 from each example included in the retraining dataset tothe site prediction model 235. Using the predicted primary site 340produced by the site prediction model 235, the model trainer 225 maycalculate another loss metric and update the weights of the siteprediction model 235 accordingly.

Upon convergence, the model trainer 225 may store and maintain the setof weights of the site prediction model 235. The convergence maycorrespond to a change in the value of the weights in the siteprediction model 235 is below a threshold value. The set of weights ofthe site prediction model 235 may be stored using one or more datastructures, such as an array, a matrix, a heap, a list, a tree, or adata object, among others. In some embodiments, the model trainer 225may store the set of weights of the site prediction model 235 in thedatabase 240.

Referring now to FIG. 5 , depicted is a block diagram of an inferenceprocess 500 for the site prediction model 235 in the system 200 fordetermining primary sites from biomedical images. The process 500 maycorrespond to or include the operations performed by the imageprocessing system 205 under the runtime mode. Under the process 500, theimaging device 210 may scan, obtain, or otherwise acquire at least oneimage 505 of at least one sample 510 from a subject 515. The image 505may be similar to the image 305 described above. The sample 510 may be atissue section taken or obtained from the subject 515. The sample 510may be obtained from at least one primary site 520 or at least onesecondary site 525 (sometimes referred herein as a metastatic site) ofthe subject 515. The sample 510 itself may have or include one or moreobjects with the conditions. For example, the tissue section of thesample 510 may contain tumorous cells or lesions thereon. The primarysite 520 may correspond to a location in the subject 515 from which thecondition originated. The secondary site 525 may correspond to alocation in the subject 515 to which the condition spread. For example,for a subject 515 with lung cancer that spread to the brain, the primarysite 520 may be a location within the lung and the secondary site 525may be a location within the brain. The image 505 may be acquired inaccordance with microscopy techniques or a histopathological imaging.Upon acquisition, the imaging device 210 may send, transmit, orotherwise provide the acquired image 505 to the imaging processingsystem 205.

The model applier 230 may in turn retrieve, receive, or otherwiseidentify the image 505 from the imaging device 210. The model applier230 may provide or feed the image 505 to the input of the siteprediction model 235. In some embodiments, the model applier 230 mayfeed the entirety of the image 505 to the site prediction model 235. Insome embodiments, the model applier 230 may select or identify one ormore tiles (e.g., similar to tiles 420) from the image 505 to input tothe site prediction model 235. Upon feeding, the model applier 230 mayprocess the input image 505 in accordance with the set of weightsarranged in the site prediction model 235 to generate at least oneoutput. The output may include one or more predicted primary sites 530.Each predicted primary site 530 may identify the primary site 520 forthe condition depicted in the sample 325 of the input image 505. Thepredicted primary site 530 may include a value (e.g., alphanumeric ornumeric) corresponding to one of the sites (e.g., organ) in the subject515. In some embodiments, the output may include a confidence score foreach predicted primary site 530 for the condition. The confidence scoremay define or indicate a degree of likelihood that the predicted primarysite 530 for the condition is the actual primary site 520 for thecondition. In some embodiments, the model applier 230 may rank thepredicted primary sites 530 by confidence scores.

With the generation, the model applier 230 may store and maintain anassociation between the image 505 and the output from the siteprediction model 235. The output may include one or more of thepredicted primary sites 430 and the corresponding confidence scores. Theassociation may be stored on the database 240 using one or more datastructures. In addition, the model applier 230 may send, transmit, orotherwise provide the output from the site prediction model 235 to thedisplay 215 for presentation or rendering. In some embodiments, themodel applier 230 may also provide the association between image 505 andthe corresponding output to the display 215. The display 215 may renderor present the output from the site prediction model 235, such as thepredicted primary sites 530 and the confidence scores. In someembodiments, the display 215 may also present the image 505 along withthe output from the site prediction model 235. The display 215 maypresent the information from the image processing system 205 in agraphical user interface. For example, the graphical user interface mayprovide the image 505, the predicted primary sites 530 ranked by theconfidence scores, and the confidence scores themselves. The graphicaluser interface may also include other information regarding the subject515 or the sample 510 from which the image 505 is obtained. In thismanner, the site prediction model 235 of the image processing system 205may be able to learn the morphological latent features from images 505to determine predicted primary sites 530 of images 505 of samples 510from the subject 515. The image 505 may have been from the secondarysite 525 or the actual primary site 520 itself.

Referring now to FIG. 6A, depicted is a flow diagram of a method 600 oftraining models to determine primary sites. The method 600 may beperformed by or implemented using the system 300 described herein inconjunction with FIGS. 2-5 or the system 700 detailed herein inconjunction in Section B. In brief overview, under the method 600, acomputing system (e.g., the image processing system 205) may identify atraining dataset (e.g., the training dataset 245) (605). The computingsystem may apply an image (e.g., the image 305) to a model (e.g., thesite prediction model 235) (610). The computing system may compare aresult (e.g., the predicted primary site 340) with a label (e.g., theprimary site label 310) (615). The computing system may update the model(620). The computing system may store weights of the model (625).

Referring now to FIG. 6B, depicted is a flow diagram of a method 650 ofapplying models to determine primary sites. The method 650 may beperformed by or implemented using the system 300 described herein inconjunction with FIGS. 2-5 or the system 700 detailed herein inconjunction in Section B. Under the method 650, a computing system(e.g., the image processing system 205) may identify an acquired image(e.g., the image 505) (655). The computing system may apply the image toa model (e.g., the site prediction model 235) (660). The computingsystem may provide a result (e.g., the predicted primary site 530)(665).

B. Computing and Network Environment

Various operations described herein can be implemented on computersystems. FIG. 7 shows a simplified block diagram of a representativeserver system 700, client computer system 714, and network 726 usable toimplement certain embodiments of the present disclosure. In variousembodiments, server system 700 or similar systems can implement servicesor servers described herein or portions thereof. Client computer system714 or similar systems can implement clients described herein. Thesystem 300 described herein can be similar to the server system 700.Server system 700 can have a modular design that incorporates a numberof modules 702 (e.g., blades in a blade server embodiment); while twomodules 702 are shown, any number can be provided. Each module 702 caninclude processing unit(s) 704 and local storage 706.

Processing unit(s) 704 can include a single processor, which can haveone or more cores, or multiple processors. In some embodiments,processing unit(s) 704 can include a general-purpose primary processoras well as one or more special-purpose co-processors such as graphicsprocessors, digital signal processors, or the like. In some embodiments,some or all processing units 704 can be implemented using customizedcircuits, such as application specific integrated circuits (ASICs) orfield programmable gate arrays (FPGAs). In some embodiments, suchintegrated circuits execute instructions that are stored on the circuititself. In other embodiments, processing unit(s) 704 can executeinstructions stored in local storage 706. Any type of processors in anycombination can be included in processing unit(s) 704.

Local storage 706 can include volatile storage media (e.g., DRAM, SRAM,SDRAM, or the like) and/or non-volatile storage media (e.g., magnetic oroptical disk, flash memory, or the like). Storage media incorporated inlocal storage 706 can be fixed, removable or upgradeable as desired.Local storage 706 can be physically or logically divided into varioussubunits such as a system memory, a read-only memory (ROM), and apermanent storage device. The system memory can be a read-and-writememory device or a volatile read-and-write memory, such as dynamicrandom-access memory. The system memory can store some or all of theinstructions and data that processing unit(s) 704 need at runtime. TheROM can store static data and instructions that are needed by processingunit(s) 704. The permanent storage device can be a non-volatileread-and-write memory device that can store instructions and data evenwhen module 702 is powered down. The term “storage medium” as usedherein includes any medium in which data can be stored indefinitely(subject to overwriting, electrical disturbance, power loss, or thelike) and does not include carrier waves and transitory electronicsignals propagating wirelessly or over wired connections.

In some embodiments, local storage 706 can store one or more softwareprograms to be executed by processing unit(s) 704, such as an operatingsystem and/or programs implementing various server functions such asfunctions of the system 500 of FIG. 5 or any other system describedherein, or any other server(s) associated with system 500 or any othersystem described herein.

“Software” refers generally to sequences of instructions that, whenexecuted by processing unit(s) 704 cause server system 700 (or portionsthereof) to perform various operations, thus defining one or morespecific machine embodiments that execute and perform the operations ofthe software programs. The instructions can be stored as firmwareresiding in read-only memory and/or program code stored in non-volatilestorage media that can be read into volatile working memory forexecution by processing unit(s) 704. Software can be implemented as asingle program or a collection of separate programs or program modulesthat interact as desired. From local storage 706 (or non-local storagedescribed below), processing unit(s) 704 can retrieve programinstructions to execute and data to process in order to execute variousoperations described above.

In some server systems 700, multiple modules 702 can be interconnectedvia a bus or other interconnect 708, forming a local area network thatsupports communication between modules 702 and other components ofserver system 700. Interconnect 708 can be implemented using varioustechnologies including server racks, hubs, routers, etc.

A wide area network (WAN) interface 710 can provide data communicationcapability between the local area network (interconnect 708) and thenetwork 726, such as the Internet. Technologies can be used, includingwired (e.g., Ethernet, IEEE 702.3 standards) and/or wirelesstechnologies (e.g., Wi-Fi, IEEE 702.11 standards).

In some embodiments, local storage 706 is intended to provide workingmemory for processing unit(s) 704, providing fast access to programsand/or data to be processed while reducing traffic on interconnect 708.Storage for larger quantities of data can be provided on the local areanetwork by one or more mass storage subsystems 712 that can be connectedto interconnect 708. Mass storage subsystem 712 can be based onmagnetic, optical, semiconductor, or other data storage media. Directattached storage, storage area networks, network-attached storage, andthe like can be used. Any data stores or other collections of datadescribed herein as being produced, consumed, or maintained by a serviceor server can be stored in mass storage subsystem 712. In someembodiments, additional data storage resources may be accessible via WANinterface 710 (potentially with increased latency).

Server system 700 can operate in response to requests received via WANinterface 710. For example, one of modules 702 can implement asupervisory function and assign discrete tasks to other modules 702 inresponse to received requests. Work allocation techniques can be used.As requests are processed, results can be returned to the requester viaWAN interface 710. Such operation can generally be automated. Further,in some embodiments, WAN interface 710 can connect multiple serversystems 700 to each other, providing scalable systems capable ofmanaging high volumes of activity. Other techniques for managing serversystems and server farms (collections of server systems that cooperate)can be used, including dynamic resource allocation and reallocation.

Server system 700 can interact with various user-owned or user-operateddevices via a wide-area network such as the Internet. An example of auser-operated device is shown in FIG. 7 as client computing system 714.Client computing system 714 can be implemented, for example, as aconsumer device such as a smartphone, other mobile phone, tabletcomputer, wearable computing device (e.g., smart watch, eyeglasses),desktop computer, laptop computer, and so on.

For example, client computing system 714 can communicate via WANinterface 710. Client computing system 714 can include computercomponents such as processing unit(s) 716, storage device 718, networkinterface 720, user input device 722, and user output device 724. Clientcomputing system 714 can be a computing device implemented in a varietyof form factors, such as a desktop computer, laptop computer, tabletcomputer, smartphone, other mobile computing device, wearable computingdevice, or the like.

Processor 716 and storage device 718 can be similar to processingunit(s) 704 and local storage 706 described above. Suitable devices canbe selected based on the demands to be placed on client computing system714; for example, client computing system 714 can be implemented as a“thin” client with limited processing capability or as a high-poweredcomputing device. Client computing system 714 can be provisioned withprogram code executable by processing unit(s) 716 to enable variousinteractions with server system 700.

Network interface 720 can provide a connection to the network 726, suchas a wide area network (e.g., the Internet) to which WAN interface 710of server system 700 is also connected. In various embodiments, networkinterface 720 can include a wired interface (e.g., Ethernet) and/or awireless interface implementing various RF data communication standardssuch as Wi-Fi, Bluetooth, or cellular data network standards (e.g., 3G,4G, LTE, etc.).

User input device 722 can include any device (or devices) via which auser can provide signals to client computing system 714; clientcomputing system 714 can interpret the signals as indicative ofparticular user requests or information. In various embodiments, userinput device 722 can include any or all of a keyboard, touch pad, touchscreen, mouse or other pointing device, scroll wheel, click wheel, dial,button, switch, keypad, microphone, and so on.

User output device 724 can include any device via which client computingsystem 714 can provide information to a user. For example, user outputdevice 724 can include a display to display images generated by ordelivered to client computing system 714. The display can incorporatevarious image generation technologies, e.g., a liquid crystal display(LCD), light-emitting diode (LED) including organic light-emittingdiodes (OLED), projection system, cathode ray tube (CRT), or the like,together with supporting electronics (e.g., digital-to-analog oranalog-to-digital converters, signal processors, or the like). Someembodiments can include a device such as a touchscreen that function asboth input and output device. In some embodiments, other user outputdevices 724 can be provided in addition to or instead of a display.Examples include indicator lights, speakers, tactile “display” devices,printers, and so on.

Some embodiments include electronic components, such as microprocessors,storage and memory that store computer program instructions in acomputer readable storage medium. Many of the features described in thisspecification can be implemented as processes that are specified as aset of program instructions encoded on a computer readable storagemedium. When these program instructions are executed by one or moreprocessing units, they cause the processing unit(s) to perform variousoperation indicated in the program instructions. Examples of programinstructions or computer code include machine code, such as is producedby a compiler, and files including higher-level code that are executedby a computer, an electronic component, or a microprocessor using aninterpreter. Through suitable programming, processing unit(s) 704 and716 can provide various functionality for server system 700 and clientcomputing system 714, including any of the functionality describedherein as being performed by a server or client, or other functionality.

It will be appreciated that server system 700 and client computingsystem 714 are illustrative and that variations and modifications arepossible. Computer systems used in connection with embodiments of thepresent disclosure can have other capabilities not specificallydescribed here. Further, while server system 700 and client computingsystem 714 are described with reference to particular blocks, it is tobe understood that these blocks are defined for convenience ofdescription and are not intended to imply a particular physicalarrangement of component parts. For instance, different blocks can bebut need not be located in the same facility, in the same server rack,or on the same motherboard. Further, the blocks need not correspond tophysically distinct components. Blocks can be configured to performvarious operations, e.g., by programming a processor or providingappropriate control circuitry, and various blocks might or might not bereconfigurable depending on how the initial configuration is obtained.Embodiments of the present disclosure can be realized in a variety ofapparatus including electronic devices implemented using any combinationof circuitry and software.

While the disclosure has been described with respect to specificembodiments, one skilled in the art will recognize that numerousmodifications are possible. Embodiments of the disclosure can berealized using a variety of computer systems and communicationtechnologies including but not limited to specific examples describedherein. Embodiments of the present disclosure can be realized using anycombination of dedicated components and/or programmable processorsand/or other programmable devices. The various processes describedherein can be implemented on the same processor or different processorsin any combination. Where components are described as being configuredto perform certain operations, such configuration can be accomplished,e.g., by designing electronic circuits to perform the operation, byprogramming programmable electronic circuits (such as microprocessors)to perform the operation, or any combination thereof. Further, while theembodiments described above may make reference to specific hardware andsoftware components, those skilled in the art will appreciate thatdifferent combinations of hardware and/or software components may alsobe used and that particular operations described as being implemented inhardware might also be implemented in software or vice versa.

Computer programs incorporating various features of the presentdisclosure may be encoded and stored on various computer readablestorage media; suitable media include magnetic disk or tape, opticalstorage media such as compact disk (CD) or DVD (digital versatile disk),flash memory, and other non-transitory media. Computer readable mediaencoded with the program code may be packaged with a compatibleelectronic device, or the program code may be provided separately fromelectronic devices (e.g., via Internet download or as a separatelypackaged computer-readable storage medium).

Thus, although the disclosure has been described with respect tospecific embodiments, it will be appreciated that the disclosure isintended to cover all modifications and equivalents within the scope ofthe following claims.

What is claimed is:
 1. A method, comprising: identifying, by a computingsystem, a first biomedical image of a first sample obtained from one ofa plurality of sites associated with a condition in a first subject, theplurality of sites including (i) a first organ from which the conditionoriginated and (ii) a second organ to which the condition spread fromthe first organ; determining, by the computing system, a sitecorresponding to the first organ for the first subject by applying thefirst biomedical image to a machine learning (ML) model, wherein the MLmodel is trained using a plurality of examples, each example of theplurality of examples including (i) a respective second biomedical imagefrom one of the plurality of sites associated with the condition in acorresponding second subject and (ii) a respective identification of thefirst organ for the corresponding second subject; and storing, by thecomputing system, for the first subject, an association between thefirst biomedical image and the determined site.
 2. The method of claim1, further comprising providing, by the computing system, an outputbased on the determined site having the association with the firstbiomedical image.
 3. The method of claim 1, further comprisingpresenting, by the computing system, information identifying thedetermine site and at least one of the first sample, the firstbiomedical image, or the first subject.
 4. The method of claim 1,wherein determining further comprises identifying a plurality ofcandidate sites for the first organ from which the condition originatedfor the first subject.
 5. The method of claim 1, wherein determiningfurther comprises generating a confidence score indicating a degree oflikelihood that the site is the first organ from which the conditionoriginated for the first subject.
 6. The method of claim 1, whereindetermining further comprises determining, for each condition of aplurality of conditions, the site for the first organ from which thecondition originated for the subject.
 7. The method of claim 1, whereinidentifying further comprises obtaining the first biomedical image ofthe first sample via a histological image preparer.
 8. A method,comprising: identifying, by a computing system, a training datasetincluding a plurality of examples, each example of the plurality ofexamples including: (i) a respective biomedical image from one of aplurality of sites associated with a condition in a correspondingsubject, the plurality of sites including (a) a first organ from whichthe condition originated and (b) a second organ to which the conditionspread from the first organ; and (ii) a respective identification of afirst organ from which the condition originated for the correspondingsubject; determining, by the computing system, for each example of theplurality of examples, a second site corresponding to the first organfor the corresponding subject by applying the first biomedical image toa machine learning (ML) model comprising a plurality of weights;comparing, by the computing system, for each example of the plurality ofexamples, the second site determined by applying the ML model with thefirst site of the respective identification in the training dataset; andupdating, by the computing system, at least one of the plurality ofweights of the ML model based on the comparison between the second sitewith the first site of the respective identification in each example ofthe plurality of examples.
 9. The method of claim 8, further comprisingreapplying, by the computing system, the respective biomedical image ofat least one example of the plurality of examples to the ML model,responsive to determining that a loss metric for the at least oneexample exceeds a threshold.
 10. The method of claim 8, whereindetermining further comprises generating, for each example of theplurality of examples, a confidence score indicating a degree oflikelihood that the second site is the first organ from which thecondition originated for the respective subject.
 11. The method of claim8, wherein comparing further comprises determining at least one lossmetric indicating a degree of deviation between the second sitedetermined by applying the ML model and the first site of the respectiveidentification in the training dataset.
 12. The method of claim 8,wherein updating further comprises updating at least one of theplurality of weights of the ML model in accordance with a classificationloss between the second site determined by applying the ML model and thefirst site of the respective identification in the training dataset. 13.The method of claim 8, wherein each example of the plurality of examplesin the training dataset further comprises a respective identification ofa third site corresponding to the second organ to which the conditionspread for the corresponding subject.
 14. The method of claim 8, furthercomprising storing, by the computing system, using one or more datastructures, the plurality of weights of the ML model to apply to anacquired biomedical image from a sample of a subject to determine thefirst organ from which the condition originated for the subject.
 15. Asystem, comprising: a computing system having one or more processorscoupled with memory, configured to: identify a first biomedical image ofa first sample obtained from one of a plurality of sites associated witha condition in a first subject, the plurality of sites including (i) afirst organ from which the condition originated and (ii) a second organto which the condition spread from the first organ; determine a sitecorresponding to the first organ for the first subject by applying thefirst biomedical image to a machine learning (ML) model, wherein the MLmodel is trained using a plurality of examples, each example of theplurality of examples including (i) a respective second biomedical imagefrom one of the plurality of sites associated with the condition in acorresponding second subject and (ii) a respective identification of thefirst organ for the corresponding second subject; and store, for thefirst subject, an association between the first biomedical image and thedetermined site.
 16. The system of claim 15, wherein the computingsystem is further configured to provide an output based on thedetermined site having the association with the first biomedical image.17. The system of claim 15, wherein the computing system is furtherconfigured to present information identifying the determine site and atleast one of the first sample, the first biomedical image, or the firstsubject.
 18. The system of claim 15, wherein the computing system isfurther configured to identify, by applying the first biomedical imageto the ML model, a plurality of candidate sites for the first organ fromwhich the condition originated for the first subject.
 19. The system ofclaim 15, wherein the computing system is further configured togenerate, by applying the first biomedical image to the ML model, aconfidence score indicating a degree of likelihood that the site is thefirst organ from which the condition originated for the first subject.20. The system of claim 15, wherein the computing system is furtherconfigured to determine, by applying the first biomedical image to theML model, for each condition of a plurality of conditions, the site forthe first organ from which the condition originated for the subject.